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ABSTRACT 

Forest Grove High School (FGHS) is a suburban school in 
Oregon that implemented an extensive remediation program to increase the 
mathematics scores of its learners taking the state-mandated competency 
examination, the Oregon Statewide Assessment Test (OSAT) . Learners who failed 
at least two of the mathematics sections of the 1998 OSAT, taken when they 
were in the eighth grade, were enrolled in a High Intensity Learning Lab 
(HILL) for remediation that included the use of the PLATO computer learning 
system. The 2000 OSAT mathematics score for sophomores assigned to the HILL 
increased from 228 to 232.1. This gain was more than two times larger than 
that of the general learner population who did not use PLATO. In the general 
learner population, among those not using PLATO, percentile ranking declined 
from 54% to 45%, but PLATO users' percentile rank remained at 41%. However, 
due to a change in the cutoff scores for the OSAT, a higher pass rate did not 
result. Only 37% of the 10th graders in Oregon had a passing 2000 OSAT 
mathematics score. A significant positive relationship was identified between 
PLATO module mastery and the 2000 OSAT test. Learners (n=38) and teachers 
generally liked the PLATO system, and the FGHS principal and PLATO instructor 
believed that PLATO contributed to learner improvement- on the OSAT. 

Appendixes contain the instructor survey results, graphs of OSAT achievement 
data, and the. learner survey results. (Contains 6 tables and 33 figures.) 
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Executive Summary 



Forest Grove High School (FGHS) is a suburban school that implemented 
an extensive remediation program to increase the math scores of their 
learners taking the state-mandated competency exam, OSAT. Learners 
who failed at least two of the math sections of the 1998 OSAT (taken 
when they were in 8th grade) were enrolled in a PLATO -supported High 
Intensity Learning Lab (HILL) for remediation. 

The 2000 OSAT math scores for all sophomores assigned to the HILL - 
including these failing learners - increased from 228 to 232.1. These 
gains were more than two times larger than those of the general learner 
population who did not use PLATO. In the general learner population, 
who did not use PLATO, percentile ranking declined from 54% to 45%. 

By contrast, the PLATO users’ percentile rank remained the same at 
41%. However, due to a change in the cutoff scores for the OSAT, a 
higher pass rate did not result. Only 37% of 10*^^ graders in Oregon had a 
passing 2000 OSAT math score. 

The purpose of this evaluation report is to describe the manner in which 
the PLATO system has been used within the Math remediation courses at 
FGHS, to examine the effectiveness of the OSAT remediation effort, and 
to suggest possible areas of improvement for future PLATO 
implementation and use. Some of the more important results of this 
evaluation include: 

• Learner OSAT scores increased significantly. 

• A significant positive relationship was identified between PLATO 
module mastery and the 2000 OSAT test. 

• Learners reported that the computer lessons made them feel more 
confident about doing well in school. 

• Learners generally agreed that PLATO was easy to use, easy to 
understand, allowed them to work at their own pace, and they 
tried hard to learn from their assigned the PLATO modules. 

• The FGHS principal and PLATO instructor believed that PLATO 
contributed to learner improvement on the OSAT courses and 
generally were positive about the software. 

Six tables are included in the evaluation which detail OSAT test results 
and instructor and learner attitude survey results. Suggestions are 
outlined for maximizing the effectiveness of future PLATO use at FGHS. 
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Introduction 



Forest Grove High School 



This report describes the PLATO-supported strategies used by Forest 
Grove High School, Forest Grove OR, to remediate learners who failed to 
earn a passing score on the math portion of the Oregon Statewide 
Assessment Test (OSAT)h Learners must pass the OSAT in 10th grade to 
receive the Certificate of Initial Mastery (CIM) in math and language arts 
(science and social studies will be phased in by 2002 and 2003 
respectively). This certificate was established in 1997. Learners in the 
class of 2002 are the second class subjected to this OSAT requirement 
and as 8th graders in 1998, over 60 percent of this class failed the OSAT. 
In 1997, nearly 60 percent of this class failed at least one part of the 
OSAT. This performance led FGHS, along with all high schools in the 
state, to devise an effective remediation plan to raise the OSAT scores for 
math and language arts on the spring 1999 and 2000 test. The 
remediation effort of the 2001 FGHS class is the focus of this evaluation 
report. This report evaluates the general effectiveness of the school’s 
remediation strategy - which included both PLATO interventions and 
traditional classroom instruction. According to the instructor, the 
HILL/ PLATO lab was the only remaining strategy to raise learner scores 
other than a tighter curriculum alignment of math curriculum. 

FGHS serves a middle and lower-middle class learner population with 
about 30% of learner population on the free or reduced federal lunch 
program. Hispanics are the main minority group, comprising about 20% 
of the learner population. 

FGHS began using PLATO in 1997-98 to develop personalized learner 
curriculum for the expressed purpose of OSAT test remediation. After the 
1997 OSAT results, all learners who failed at least one part of the test 
were required to enroll in the newly created PLATO High Intensity 
Learning Lab for remediation. This effort continued in the following 
school year, 1998-99. Learners who did not have a composite score of 
237 or higher on their 1998 8^^ grade OSAT math multiple-choice test 
were assigned to the lab. This criterion was derived by targeting learners 
who fell within 2 standard deviations from the 8th grade passing score of 
231 (at the 47‘^ percentile ranking). The school targeted learners, who 
had scores of 237 (67‘^ percentile) or lower. The lower end (225) was 
dropped so that all learners below the 237 whose scheduling allowed (by 
being enrolled in study hall) were included in the HILL program. 



1 This report is an update and expansion of Bob Hannalin and David 
Opperman’s earlier report on Forest Grove High School. 
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PLATO Pathways was implemented to meet the following instructional 
needs: 

• Align the curriculum to the Oregon Content Standards 

• Address the remediation needs of each learner, since each comes 
into the program with a wide range of previous experiences and 
skill levels 

• Enable instructors to establish learning programs that present a 
variety of computer-based modules correlated to math and 
language arts 

• Provide a motivational way to learn core subject material 

• Provide individual assessment and tracking 

The purpose of this evaluation report is to describe the manner in which 
the PLATO Pathways program has been used in High Intensity Learning 
Lab (HILL) at FGHS, to examine the effectiveness of the OSAT 
remediation effort, and to suggest possible areas of improvement for 
future PLATO implementation and use. This evaluation also will attempt 
to determine the relationship between the completion of aligned 
computer assisted instruction and improved performance on the math 
portion of the OSAT. 
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Program Description 

Learners. Approximately 70% of the FGHS class of 2001 had failed one or 
both parts of the 1997 OSAT and were automatically enrolled in the High 
Intensity Learning Lab. Approximately 60% of the FGHS class of 2002 
had failed the math portion of the 1998 OSAT and were automatically 
enrolled in the High Intensity Learning Lab. 

Program Goals. The goal for using PLATO, and the focus of this report, 
was to help increase the passing rate for learners who have already failed 
the math components of the OSAT. 

According to Mr. Robert Schlegel, Principal - FGHS and Dr. Jan Jones, 
Director of Curriculum and Instruction - Forest Grove School District, it 
was originally decided to assign curriculum in the lab according to 
individual learner achievement. Curriculum would be assigned to 
learners according to the results of the 8th grade OSAT scores for math 
and reading/literature. These two tests were selected as the initial focus 
of the lab with the understanding that supplemental curriculum for 
performance-based goals such as writing and mathematical problem 
solving would be given future consideration. Other learners were 
accepted to enroll in the lab; it was not exclusively used for OSAT 
remediation. 

The next step in the project was to create an alignment between the 
Oregon content standards and the PLATO curriculum. This involved 
dissecting the Oregon benchmarks into smaller components and 
matching the PLATO learning objectives. Each benchmark for math was 
created as a course containing an average of twenty (20) modules. 

A PLATO module is based on a single learning objective containing three 
activities; a tutorial that teaches the concept, an application that 
provides additional practice and one or more mastery tests, which must 
be eighty percent (80%) correct in order to demonstrate mastery. 

Learners not achieving eighty percent (80%) mastery are guided back 
through the tutorial before they can retake the mastery tests. No 
problem-solving activities (PSA’s) from the Math Problem Solving 
curriculum, and no non-PLATO learning activities were implemented in 
this program, though it did not displace their normally scheduled math 
class. All learners in the class of 2002 whose scheduling allowed were 
assigned the aligned course work for each benchmark in which they had 
received a failing score on the 1998 OSAT (taken in 8th grade). 

Learners were scheduled to attend a forty-five (45) minute session in the 
lab during their ninety (90) -minute resource block, which occurred every 
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other school day according to the block schedule. The HILL program 
implements each aspect of PLATO’s Skills Development Model. ^ 

Instructor Characteristics and Role in Program. Mr. David Opperman was 
the instructor and taught in the High Intensity Learning Lab. The task of 
the instructor was to design and align the curriculum with state 
benchmarks and FGHS’s overall goals. His role was to facilitate, to the 
extent possible, each learner’s curriculum, answer content questions, 
manage module mastery and testing, and troubleshoot technical 
problems in the lab. Mr. Opperman continually interacted with HILL 
learners individually. He helped problem-solve individual math problems 
and provided motivational support. He occasionally added PLATO 
modules for individual learners if he thought a particular content area 
needed to be mastered by a learner. 

PLATO Implementation Description 

A concerted effort to use PLATO to increase the OSAT passing rate was 
undertaken in the 1997-98 academic year and continued in the 1998-99 
academic year. Learners in need of remediation were enrolled in the 
HILL and then placed in PLATO modules. Once placed in the OSAT- 
aligned PLATO curriculum, learners then worked at their own pace 
through the curriculum, advancing to the next module after passing the 
module mastery test. Mr. Opperman facilitated learner progress 
individually. 

Content standards in Oregon are organized into specific goals called 
benchmarks, which define what is to be achieved by learners at grades 3, 
5, 8 and 10. The benchmarks for mathematics are: 
calculation/estimation, measurement, probability and statistics, 
algebraic relations, geometry and mathematical problem solving. These 
benchmarks are measured by the multiple choice OSAT, except for 
mathematical problem solving, which is measured by a performance- 
based assessment. 

In 1997-98, the first year of the FGHS remediation effort, the curriculum 
emphasis in the HILL was equally split between math and language arts. 
But the strategy shifted in the second year to place greater emphasis on 
math because Mr. Schlegel and faculty were concerned with the low 
math scores of the 1997-98 sophomores (rising seniors) in the spring 
1998 OSAT. This effort continued with the learners in the class of 2002. 



^ For more information about this model, see Foshay, R., Instructional models: Four ways to 
integrate PLATO into the curriculum. Technical Paper #6: Edina, MN: PLATO Learning, Inc., 
2000 . 
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Evaluation Design 



The present evaluation examines elements of the implementation using a 
design which is somewhat consistent with both a modified Masteiy- 
Based Program Effectiveness design and an Affective Outcomes design. ^ 
PLATO module-mastery and time-in-program data were analyzed. In 
addition, the learner OSAT scores from 1998 and 2000 were examined, 
as well as program description and affective outcome data were collected. 

Learners who met the following criteria were included in the analysis: 

• Learners who did not meet at least two benchmarks for the math 
portions of the OSAT 

• Learners who were assigned at least forty modules in math 

• Learners who completed 70% of the assigned course work in math. 

In 1997, learners originally were assigned to the HILL for both math and 
English based on their 8th grade OSAT results, but after the first year, 
emphasis was shifted away from English in favor in math in response to 
poor math scores school-wide. Thus, only the math PLATO module data 
and math OSAT results were analyzed in this quantitative portion of this 
evaluation. Learner OSAT scores were examined for all Forest Grove 10th 
graders for whom scores were available for both the 8th and 10th grades. 

Affective outcomes were measured with questionnaires completed by 
FGHS faculty and learners. In addition, a telephone interview was 
conducted with the principal, Mr. Robert Schlegel. 

The current evaluation primarily examined learners in the class of 2002 
(rising juniors) who were enrolled in the HILL. Most of this group of 
learners participated in the concerted remediation effort with extensive 
use of PLATO; the rest of the group used PLATO for other purposes but 
s till took the OSATs in both years under examination. Examining this 
group of learners offers the most promise for determining how successful 
the OSAT remediation effort has been, and to what extent PLATO has 
contributed in that success. 

The number of learners enrolled in the HILL was approximately 200 for 
both 1997-98 and 1998-99. The group of Forest Grove learners who 
were not enrolled in HILL provides somewhat of a comparison group. By 
comparing these two groups, the design requires that their results differ 
from each other by a certain amount in order to determine the difference 
is not due to chance alone. 



^ Foshay, R., Guidelines for Evaluating Programs Using PLATO. Technical Paper #2: Edina, MN: 
TRO Learning, Inc., 1994. 
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The PLATO module data (module mastery, time in program) for learners 
who worked in the HILL were correlated to the OSAT test scores. The 
evaluation seeks to examine relationships among several variables as 
well as describe a rich picture of participant attitudes and beliefs . 
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Data Analysis 



Results of instructor and learner surveys are reported. For open-ended 
survey items, similar responses are summarized and reported. In 
reporting the interview results, common threads and main ideas were 
collapsed and summarized. In the quantitative analysis, correlations 
were performed and reported at the .05 alpha level of significance. That 
is, there had to be a 95% likelihood that the correlations were not due to 
random variation, in order for them to be considered in the conclusions. 

Differences in OSAT scores were analyzed using a paired sample t-test, 
again at the .05 level. In other words, learner gain scores were examined 
to determine if the gains were likely due to the FGHS’s HILL intervention 
or are a result of random fluctuation. The analysis is focused on those 
sophomores whose OSAT scores were available from both 8th and 10th 
grades. 

Procedures for data collection 

The evaluator used the PLATO site overview questions'^ to structure the 
phone interviews, and then allowed the inquiry to be guided by the 
concerns and perspectives of the participants. Mr. Opperman 
administered the learner surveys to the classes that used PLATO, 
completed the instructor survey himself, and forwarded all data to the 
evaluator. Unfortunately, only 38 learners returned completed learner 
surveys because the surveys were administered during the last days of 
the school year. Mr. Opperman also provided comprehensive PLATO 
records of all HILL learners, and learner scores for the past two OSATs, 
which were used in the current analysis. 

Results 

The results are organized into two sections, OSAT scores and 
Attitudes/Beliefs. The OSAT scores section examines the OSAT scores for 
the rising juniors who participated in the HILL over the last two years. 
These learner scores were analyzed for tests administered in 1998 and 
2000. In addition, Pearson product moment correlations were calculated 
between the PLATO module data and the math components of the OSAT 
administered in Spring 2000. Graphs supporting this analysis are in 
Appendix B. The Attitudes/ Beliefs section presents the attitude 
questionnaire data for the instructor (Mr. Opperman) and learners, and 
descriptions of the telephone interviews with Mr. Schlegel and Mr. 
Opperman. The data tabulation and graphs for this section are in 
Appendix A and Appendix C. 



^ Part of guidelines to evaluators - ed. 
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Table la and lb displays learner mean scores for the math component of 
the OSATs administered in 1998 and 2000. Test scores from these dates 
are important to examine because the 1998 scores predate all of the 
OSAT remediation efforts initiated by FGHS, and the 2000 test was 
administered after the OSAT remediation courses were in place for 
almost two years. PLATO users include learners who participated in the 
OSAT remediation courses as freshmen and sophomores. Non-PLATO 
users include all other Forest Grove sophomores who were not assigned 
to the HILL and for whom there were OSAT scores available for both 
1998 and 2000. 
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Table la. Mean OSAT 1998 and 2000 scores 



Users 




OSAT scores 
1998 


OSAT scores 
2000 


PLATO users 


M 


228.08 


232.16 




SD 


(7.93) 


(9.36) 




N 


146 


146 


Non-PLATO 


M 


232.13 


233.80 


users 


SD 


(12.64) 


(14.45) 




N 


85 


85 


Total FGHS 


M 


229.57 


232.76 


learner scores 


SD 


(10.09) 


(11.49) 




N 


231 


231 
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Table lb. Mean OSAT 1998 vs. 2000 gain scores 



Users 




OSAT score - gain 


PLATO users 


M 


4.08 




SD 


(5.4) 




N 


146 


Non-PLATO users 


M 


1.67 




SD 


(6.33) 




N 


85 


Total 


M 


3.19 




SD 


(5.86) 




N 


231 



Paired-sample i-tests revealed that learner OSAT scores on the 2000 test 
increased significantly over the 1998 test for both PLATO users and non- 
PLATO users. PLATO users increased from a mean score of 228.08 to a 
mean score of 232. 16, for a gain score of 4.08 points, t(dfl45)=-9. 125 
p<.000 (effect size = .19). Non-PLATO users increased from a mean score 
of 232.13 to a 233.8, for a gain score of 1.67 points, t(df84) = -2.434 
p<.017 (effect size = .19). Gains on the 2000 OSAT of the HILL learners 
were nearly three times larger than other learners (see table lb). 

Because the passing score was raised from 231 to 239, fewer learners 
met the standard in 2000. In 1998 as eighth graders, 91 of the total 231 
learners for whom OSAT scores are available, met the 231 OSAT 
minimum standard - about a 39 % pass rate. Of those 140 learners who 
failed, 103 were assigned to the HILL and the other 37 took the regular 
math curriculum. In 2000, only 65 of the 231 learners for whom 2000 
OSAT scores are available, met the higher 239 OSAT minimum standard 
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- about a 28 % pass rate. Of those 65 passing learners, 30 had worked in 
HILL and the other 35 had worked in the regular math curriculum. 

Running an one-way analysis of variance, there was a significant 
difference between PLATO learners’ post-OSAT scores and non-PLATO 
learners’ post-OSAT scores F(48,182) = 1.467, p<.038. Apparently, 

FGHS learners, who were enrolled in the HILL program (146 learners), 
significantly improved their OSAT 2000 scores, compared to their non- 
HILL counterparts (85 learners). 

Strands data results 

The following tables report the average gains for PLATO and non-PLATO 
learners for the five strands (i.e.. Calculation/ Estimation, Algebraic 
Relations, Geometry, Measurement, and Statistics). Learners were 
counted as PLATO users if they had completed at least 70% of the 
assigned strands. HILL users in the Calculation/ Estimation strand 
gained more than two times that of their counterparts. HILL users in the 
Algebraic Relations strand gained more than four times than their 
counterparts. HILL users in the Geometry strand gained nearly eight 
times more than their counterparts. HILL users in the Measurement 
strand gained over six times more than their counterparts. HILL users in 
the Statistics strand gained nearly eight times more than their 
counterparts. See table 2 for more information on these gains, and 
Appendix B for graphs of the data. 
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Table 2. OSAT gains - PLATO and non-PLATO users 



Users 




OSAT scores 
2000 - gain 


PLATO users 


M 


4.06 


Calculation / Estimation 


SD 


(10.59) 




N 


109 


Non-PLATO users 


M 


1.94 


Calculation / Estimation 


SD 


(12.82) 




N 


122 


PLATO users 


M 


3.15 


Algebraic Relations 


SD 


(11.2) 




N 


86 


Non-PLATO users 


M 


-.48 


Algebraic Relations 


SD 


(14.02) 




N 


145 
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Table 2 (cont’d). OSAT gains - PLATO and non-PLATO users 



Users 




OSAT scores 
2000 - gain 


PLATO users 


M 


7.93 


Geometry 


SD 


(12.71) 




N 


84 


Non-PLATO users 


M 


1.01 


Geometry 


SD 


(12.73) 




N 


149 


PLATO users 


M 


6.98 


Measurement 


SD 


(11.07) 




N 


48 


Non-PLATO users 


M 


1.57 


Measurement 


SD 


(12.62) 




N 


183 
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Table 2 (cont’d). OSAT gains - PLATO and non-PLATO users 



Users 




OSAT scores 
2000 - gain 


PLATO users 


M 


8.54 


Statistics 


SD 


(10.25) 




N 


56 


Non-PLATO users 


M 


1.63 


Statistics 


SD 


(12.29) 




N 


175 



T-Tests - Strands 

Paired- sample t-tests revealed that learner OSAT sub-scale score gains 
on the 2000 test increased significantly over the 1998 test for PLATO 
users for the following HILL strands: 

• PLATO users for the Calculation and Estimation strand increased from 
a mean score of 228.02 to a mean score of 232.08, for a gain score of 
4.06 points, t(df 108)=-4.008 p<.000. 

• PLATO users for the Algebraic Relations strand increased from a mean 
score of 229.55 to a mean score of 232.70, for a gain score of 3. 15 
points, t(df 85)= -2.607 p<.011. 

• PLATO users for the Geometry strand increased from a mean score of 
228.33 to a mean score of 236.26, for a gain score of 7.93 points, t(df 
83)= -5.718 p<.000. 

• PLATO users for the Measurement strand increased from a mean score 
of 224 to a mean score of 230.98, for a gain score of 6.98 points, t(df 47)= 
-4.368 p<.000. 

• PLATO users for the Statistics strand increased from a mean score of 
225.21 to a mean score of 233.75, for a gain score of 8.54 points, t(df 
55)= -6.229 p<.000. 
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Paired-sample t-tests also were run for non-PLATO users for each of the 
strands. However, there were no significant results for these t-tests, 
indicating that non-PLATO users made no significant gains over the two 
years in any of the strands. 

In addition to the HlLL’s five OSAT strands, data was analyzed from 
PLATO learners who completed advanced PLATO modules, such as 
Intermediate Algebra, Geometry 2 and a combination of these modules. 
Intermediate Algebra and Geometry 2 learners were counted as PLATO 
users if they completed at least 70% of the assigned PLATO modules. See 
table 3 for more information on these gains. 



Table 3. Intermediate Algebra and Geometry 2 PLATO and non-PLATO gains 



Users 




OSAT scores 
2000 - gain 


PLATO users 


M 


5.95 


Intermediate Algebra 


SD 


(5.23) 




N 


41 


Non-PLATO users 


M 


2.59 


Intermediate Algebra^ 


SD 


(5.83) 




N 


190 



5 Non-PLATO users are those learners who did not participate in HILL. 
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Table 3 (cont’d). Intermediate Algebra and Geometry 2 PLATO and non-PLATO 
gains 



Users 




OSAT scores 
2000 - gain 


PLATO users 


M 


6.29 


Geometiy2 


SD 


(4.84) 




N 


41 


Non-PLATO users 


M 


2.55 


Geometiy2 


SD 


(5.86) 




N 


191 


PLATO users 


M 


5.58 


Intermediate Algebra/ 


SD 


(4.87) 


Geometiy2 


N 


53 


Non-PLATO users 


M 


2.52 


Intermediate Algebra/ 


SD 


(5.95) 


Geometiy2 


N 


179 
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Table 3 (cont’d). Intermediate Algebra and Geometry 2 PLATO and non-PLATO 
gains 



Users 




OSAT scores 
1998 


OSAT scores 
2000 


PLATO users 


M 


230.76 


236.71 


Intermediate 


SD 


(6.83) 


(8.88) 


Algebra 


N 


41 


41 


Non-PLATO users 


M 


229.32 


231.91 


Intermediate 


SD 


(10.66) 


(11.83) 


Algebra 


N 


190 


190 


PLATO users 


M 


231.32 


237.61 


Geometry2 


SD 


(7.31) 


(9.12) 




N 


41 


41 


Non-PLATO users 


M 


229.20 


231.75 


Geometry2 


SD 


(10.54) 


(11.68) 




N 


191 


191 



Forest Grove High School 



Page 21 




21 



Table 3 (cont’d). Intermediate Algebra and Geometry 2 PLATO and non-PLATO 
gains 



Users 




OSAT scores 
1998 


OSAT scores 
2000 


PLATO users 


M 


230.28 


235.87 


Intermediate 


SD 


(7.22) 


(9.18) 


Algebra/Geometiy2 


N 


53 


53 


Non-PLATO users 


M 


229.31 


231.83 


Intermediate 


SD 


(10.79) 


(11.97) 


Algebra / Geometiy2 


N 


179 


179 



T-Tests: Intermediate Algebra and Geometry 2 

Paired-sample t-tests revealed that learner OSAT sub-scale scores on the 
2000 test increased significantly over the 1998 test for PLATO users and 
non-PLATO users for Intermediate Algebra, Geometry2 and a 
combination of these modules, including: 

• PLATO users for the Intermediate Algebra modules increased from 
a mean score of 230.76 to a mean score of 236.71, for a gain score 
of 5.95 points, t(df 40)= -7.280 p<.000. 

• PLATO users for the Geometry2 modules increased from a mean 
score of 231.32 to a mean score of 237.61, for a gain score of 6.29 
points, t(df 40)= -8.318 p<.000. 

• PLATO users for the Intermediate Algebra/Geometry2 modules 
increased from a mean score of 230.28 to a mean score of 235.87, 
for a gain score of 5.58 points, t(df 52)= -8.357 p<.000. 

• Non-PLATO users for the Intermediate Algebra modules increased 
from a mean score of 229.32 to a mean score of 231.91, for a gain 
score of 2.59 points, t(df 189)= -6.136 p<.000. 
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• Non-PLATO users for the Geometiy2 modules increased from a 
mean score of 229.20 to a mean score of 231.75, for a gain score of 
2.55 points, t(df 190)= -6.027 p<.000. 

For each of the five strands (Calculation/Estimation, Algebraic Relations, 
Geometry, Measurement, and Statistics), Intermediate Algebra and 
Geometiy2, there were the following significant differences in OSAT post- 
scores between PLATO and non-PLATO learners, including: 

• Algebraic relations; F(21, 209) = 2.085, p<.005 

• Geometry; F(25, 207) = 1.535, p<.056 

• Measurement; F(23, 153) = 1.690, p<.033 

• Statistics; F(36, 194) = 1.912, p<.003 

• Intermediate Algebra/ Geometry2; F(44, 187) = 1.919, p<.001 

Correlations between PLATO Module Mastery and 1999 OSAT Math 
Scores 

Many factors influence the relationship of PLATO usage data and 
achievement, so correlations of these data with gains are often difficult to 
interpret. For completeness, however, they are reported here in tables 4 
and 5. 

Table 4. Correlation with OSAT Scores 



PLATO module 


N 


r 


Overall 


146 


.19* 


Geometry 


84 


.267* 


Intermediate 
Algebra/ Geometry2 


53 


.281* 



* Correlation is significant at the 0.05 level (2 -tailed). 
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Table 5. Correlations between PLATO Module Mastery and OSAT gain 



PLATO module 


N 


r 


Overall 


146 


.307** 


Geometry 


84 


.25* 


Intermediate Algebra 


41 


.34* 



* Correlation is significant at the 0.05 level (2-tailed). 
** Correlation is significant at the 0.01 level (2-tailed). 



Table 6 (next page) displays learner OSAT percentile rankings for math 
scores for 8th Grade (1998) and 10th Grade (2000). These data are 
interesting because they demonstrate a difference between PLATO users’ 
2000 OSAT improvement and the rest of the school’s OSAT 2000 
improvement. Somewhat masked by the OSAT raw scores and the drop 
in the 2000 pass rate (with the cutoff score increased to 239), the 
percentile ranking of the learners assigned to the HILL stayed the same 
from 41 % in 1998 to 41 % in 1999 t(dfl45)=-9.125 p<.000 (effect size = 
.19; average score = 232). Conversely, non-PLATO users and the entire 
FGHS learner body percentile ranking dropped. Non-PLATO users’ 
percentile ranking dropped from 54% in 1998 to 45% in 2000 t(df84) = - 
2.434 p<.017; whereas the entire FGHS learner body’s percentile ranking 
dropped from 45% in 1998 to 41% in 2000 t(df(230) = -8.275) p<.000; 
average score = 232. For more information, see figures 10-11. 

Essentially, the performance of the learners in the HILL cannot be held 
responsible for the overall FGHS learner body decline in the 2000 OSAT. 
This is important because it shows that compared to the rest of the state, 
the learners assigned to the HILL improved considerably - despite the 
high fail rate. It also establishes that the OSAT test administered to 
sophomores in 2000 with a pass score of 239 was relatively more difficult 
than the 1998 test. Thus, it is important to consider percentiles along 
with raw scores and pass/ fail data in evaluating and planning 
remediation strategies. 
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Table 6. OSAT Percentile Rankings for Math Scores for 8th Grade (1998) and 10th 
Grade (2000) 



Users 




OSAT scores 
1998 


OSAT scores 
2000 


PLATO users 


M 


41% 


41% 




SD 


(7.93) 


(9.36) 




N 


146 


146 


Non-PLATO 


M 


54% 


45% 


users 


SD 


(12.64) 


(14.45) 




N 


85 


85 


Total FGHS 


M 


44% 


41% 


learner scores 


SD 


(10.09) 


(11.49) 




N 


231 


231 



A ttitudinal/beliefs 

Instructor. Table 7 (see Appendix A) displays the responses to the 
instructor survey completed by Mr. Opperman in July 2001. It includes: 
Part 1 - instructors’ agreement or disagreement with different PLATO 
features; and Part 2 - instructors’ descriptions of how often they perform 
certain priming and instructional activities in support of PLATO. Mr. 
Opperman reported that he has used computers (including non-PLATO) 
in his teaching for eight years. His open-ended responses are reported in 
Table 8 (in Appendix A). 

Learners. Mean score responses to the Likert items in the learner survey 
are displayed in Table 9 (see Appendix C). These survey items are based 
on the respondents’ agreement or disagreement with different PLATO 
features (Strongly Agree = 5, Agree = 4, 3 = Neutral, 2 = Disagree, 1 = 
Strongly Disagree). Part 2 of the learner survey asked learners to 
describe what they liked and disliked about PLATO and is summarized in 
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Table 10 (in Appendix C). Figures 13 - 33 include bar charts displaying 
frequency distribution by response for each item on the learner survey. 

Overall, learners’ average attitude scores were positive. Learners 
reported little computer anxiety or difficulty working with computers (see 
responses to items 1-4). Responses to items addressing self-efficacy 
(items, 12, 13, 16, and 20) were average. Only one question received an 
average of less than 3. The question, “the computer makes me nervous” 
received a 2.08. 

Principal. Mr. Robert Schlegel, FGHS principal, was interviewed during 
late July, 2001. Results of this interview are summarized and analyzed 
in this section. The interview lasted approximately forty-five minutes. 

The interview covered a range of topics related to the implementation of 
PLATO in the HILL program. 

FGHS’ original purpose of using PLATO was to improve learners’ math 
and language arts scores. Mr. Schlegel is veiy pleased with PLATO and 
noted significant scores and overall improvement. The HILL program is 
fully attended by FGHS learners. Honor learners are the only learners 
who do not fully participate. Learners “see it as their normal routine. 
There is no griping about being in the lab” from learners or parents. 

Though improving learners’ math scores is still the overriding purpose, 
there is interest in “expanding its [PLATO] use.” Mr. Schlegel commented 
that they are “underutilizing [the] software’s capacity.” He is interested 
in knowing more about other software packages, such as reading, 
history, science, etc. He also expressed an interest in finding out more 
and possibly using PLATO ’s Math Problem Solving package. 

Some of Mr. Schlegel’s positive comments focused on Mr. Dave 
Opperman’s management of the HILL. He commented that Dave has a 
special combination of interpersonal and technical skills to make HILL a 
success. He “meets these expectations in spades.” He is confident that 
Dave would do all he could do to accelerate learners’ individual learning 
during HILL. Dave may expand learners’ learning profile depending on 
their math expertise. He is “a master” of organizing HILL activities with 
instructors and their curriculum. He definitely has credibility with 
instructors from a variety of disciplines. FGHS instructors are impressed 
with PLATO because of Dave. 

Because of this confidence and enthusiasm about PLATO, there is a new 
PLATO initiative at FGHS. This school is developing a credit recovery 
program with several disciplines. During the fall semester, learners who 
have existing low scores from a previous class, now can redo their grade 
using PLATO. This will take place during after the regular school hours. 
There also is interest in implementing PLATO at the middle school. 
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School officials noticed a gap in math scores for the 6*^ and 7 ^^ 
grades. Mr. Schlegel wonders if PLATO can help with these gaps. 
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Discussion and recommendations 



One of the most striking statistics is the gain on the overall OSAT and 
the five strands for PLATO learners compared to non-PLATO 
counterparts. In each scenario (i.e., overall OSAT score and the five 
strands), PLATO learners made considerable gains. This is particularly 
evident in the higher levels of math (i.e., Geometry, Measurement, 
Statistics and Geometiy2). Despite a higher cutoff score of 239 for the 
2000 OSAT exam, HILL learners’ percentile ranking remained the same 
at 41% whereas non-HILL learners and the entire FGHS learner body 
rankings declined. It is interesting to note that HILL learners, non-HILL 
learners and entire FGHS learners outperformed the state’s passing rate. 
Only 37% of 10‘*^ graders in Oregon high schools passed the 2000 OSAT. 
Although only 29% of FGHS learners passed the 2000 OSAT^ and the 
overall lack of pass rate improvement should remain an issue for the 
school, it is important to note that the proportion of HILL learners 
passing the test just about equaled the non-HILL learners, even though 
the HILL learners represent the bottom two standard deviations of the 
population. Non-PLATO users most likely did not encounter a ceiling 
effect”^. The highest OSAT 2000 score for a non-PLATO user was 267 
(99%) and the average passing score for non-PLATO users was 247.88 
(82%). For the HILL learners, the highest passing score was 260. The 
average passing score was 244. The average score for all HILL learners 
was 232. Obviously, HILL learners have made dramatic strides in their 
math skills as demonstrated in their OSAT scores. In effect, they 
“caught up” with their non-HILL peers. Again, no ceiling effect is evident. 
Quite possibly an extended study of HILL learners’ usage of PLATO needs 
to go beyond this two-year duration. 

From the learners’ attitude survey, there appears to be overall 
satisfaction and confidence. Compared to Hannafin and Opperman’s 
1999 report, there are many similarities in responses. For instance, for 
the question, “Getting to my lesson is easy”, an average response for 
2002 respondents was 4.3, whereas the average score for 2001 
respondents was 4.64. Average score for 2002 respondents for the 
question, “My instructor helps me see the connection between what I'm 
studying on and off the computer,” was 4.00; their 2001 counterparts’ 
average score was 3.98. 2002 and 2001 respondents expressed their 
confidence in doing their schoolwork. The average score for 2002 
respondents for the question, “The computer lessons make me feel more 
confident about doing well in school” was 3.38 and the average score for 



^ In 2001, 38% of FGHS learners passed the OSAT math test. 

^ A ceiling effect occurs when learners’ achievement levels exceed what the test can measure. 
This can have the effect of depressing measured learning gains erroneously. 



Forest Grove High School 



Page 28 



2001 respondents was 3.51. One learner writes “I am doing a lot better 
in Math!!!” There are some concerns about respondents’ “boring” 
statements that voice their displeasure towards PLATO. One respondent 
suggested making PLATO more interactive. This respondent remarked, 
“it is not adjusted to the person; it is just one test for everyone.” 

This voiced suggestion and concerns point to possible avenues of 
improvement for the HILL program. One cannot expect PLATO to cover 
all of the state’s curriculum standards — it cannot be a “magic bullet” or a 
“sole solution.” Therefore, some additional PLATO and non-PLATO 
activities may need to be included in the instruction. One possible 
additional PLATO activity could be the implementation of Problem 
Solving Activities (PSA’s) included in PLATO Math Problem Solving 
curriculum. PSA’s are based upon principles for teaching problem 
solving.* This would add variety and interactivity to the curriculum, and 
strengthen the curriculum’s emphasis on problem-solving. Because of 
FGHS’ enthusiasm for PLATO products, it appears that both Mr. Schlegel 
and Mr. Opperman would be interested in this additional package. 

It might also be possible to use PLATO ’s diagnostic and practice tests to 
further individualize the curriculum. However, care must be taken to 
maintain the strength of the present alignment to state standards, a 
major advantage of the present program. 

Another recommendation is to reinstate FGHS usage of the PLATO 
language arts curriculum^. There is a known correlation between reading 
ability and math achievement, which has nothing to do with technology. 
FGHS may consider expanding the HILL to include reading, or at least 
screening for reading level, is an additional recommendation for 
improvement. In the future, HILL learners , who read below grade level 
could first work on improving their reading, then work on their math. 



® For more information about these principles, please refer to Foshay, R. and Kirkley, J., 
Principles for Teaching Problem Solving. Technical Paper #4: Edina, MN: TRO Learning, Inc., 1998. 

^ In 2001, PLATO released an entirely new reading curriculum. 
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Appendix A; Instructor Survey Results 

Table 7: Instructor Survey Response by Item 



Part I Directions: We would like to know how you felt about your 
experience teaching with the PLATO® system. For each of the 
statements below, please check {y) the box under: 

SA if you strongly agree 
A if you agree 

N if you neither agree nor disagree 

D if you disagree 

SD if you strongly disagree 



Question SA AND SD 



1 


The PLATO course content includes what my 
students need to learn about the topics taught. 


□ 


X 


□ 


□ 


□ 


2 


The PLATO course objectives correspond to those 
for my course. 


□ 


X 


□ 


□ 


□ 


3 


The PLATO course content corresponds to the 
content of the standard end-of-course test we use. 


□ 


X 


□ 


□ 


□ 


4 


Content seemed generally free of errors and 
inaccuracies. 


□ 


X 


□ 


□ 


□ 


5 


Content was generally up-to-date. 


□ 


X 


□ 


□ 


□ 


6 


Quality and style of instruction was consistent 
throughout the curriculum. 


□ 


X 


□ 


□ 


□ 


7 


Students generally understood the explanations. 


□ 


X 


□ 


□ 


□ 


8 


There was adequate depth in exercises and tests. 


□ 


X 


□ 


□ 


□ 


9 


Tests, application /drill lessons, and tutorials 
corresponded to the objectives in the Instructor 
Guides. 


X 


□ 


□ 


□ 


□ 


10 


Tutorials involved the students through frequent 
questions, answers and feedback, rather than just 
reading and mouse-clicks to go on. 


X 


□ 


□ 


□ 


□ 


11 


Software was generally free of bugs and errors. 


□ 


X 


□ 


□ 


□ 
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Question 


SA 


A 


N 


D 


SD 


12 


All courseware used consistent keystrokes and 
display style. 


X 


□ 


□ 


□ 


□ 


13 


Color was used appropriately. 


□ 


X 


□ 


□ 


□ 


14 


Graphics were used appropriately. 


□ 


X 


□ 


□ 


□ 


15 


Screens were consistently readable. 


X 


□ 


□ 


□ 


□ 


16 


I was able to use student progress reports to 
identify students needing my attention. 


X 


□ 


□ 


□ 


□ 


17 


I was able to spend time in one-on-one tutoring 
and counseling while students used PLATO. 


X 


□ 


□ 


□ 


□ 


18 


I was able to make appropriate individual student 
assignments on the system. 


X 


□ 


□ 


□ 


□ 


19 


My students were scheduled to use PLATO for as 
much time as they needed. 


□ 


□ 


X 


□ 


□ 


20 


I was able to relate what the students studied on 
PLATO to what they studied in other activities. 


X 


□ 


□ 


□ 


□ 


21 


In general, my students respond well to the 
PLATO system. 


X 


□ 


□ 


□ 


□ 


22 


My students rarely seemed confused or "trapped" 
by the system. 


□ 


X 


□ 


□ 


□ 


23 


My students respond well to the PLATO system. 


X 


□ 


□ 


□ 


□ 


24 


I find working with the computer is generally a 
productive, rather than frustrating, experience. 


X 


□ 


□ 


□ 


□ 


25 


I enjoy working with the PLATO computer system. 


X 


□ 


□ 


□ 


□ 


26 


The PLATO system plays a useful role in my 
teaching. 


X 


□ 


□ 


□ 


□ 


27 


I was adequately trained to operate the PLATO 
system. 


X 


□ 


□ 


□ 


□ 


28 


I would like more training on how to use PLATO to 


X 


□ 


□ 


□ 


□ 



best advantage in my teaching. 
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Table 7 (cont’d) 

Part II Directions: 



Please rate how often you performed the following activities in class 
before your students used PLATO. Circle your responses using the 
following scale: 

5 = Before or after each computer session 
4 = Before or after most computer sessions 
3 = Occasionally, before or after a new unit or lesson 
2 = At the beginning of each semester or marking period 
1 = Maybe one time during the year 
0 = Never 





Item 


Rating 


29 


Articulated to the student(s) in some way those prerequisite skills, 
knowledge, or attitudes needed to fully succeed with their newly 
assigned PLATO modules. 


^ 4 3 2 1 0 


30 


Helped the students relate what they were about to learn in their 
PLATO assignments to their own personal previous experiences. 


^ 4 3 2 1 0 


31 


Described to the students the specific objectives they were going to 
learn within their assigned PLATO courses or modules. 


^4 3 210 


32 


Explained to the students how the skills and knowledge learned 
within their assigned PLATO modules fit into the overall course or 
lesson goals. 


5 ^3 210 


33 


Clearly identified to the students the rewards and incentives for 
trying hard and doing well within the PLATO system. 


5 4 3 2 1 0 


34 


Explained to the students specific procedures for getting support if 
they didn't understand something they were trying to learn within 
the PLATO system. 


5 _4 3 2 1 0 
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Table 8: Summary of Instructor Open-Ended Survey Responses 



Part III Directions: Please write your response to each question in the 
space provided. 

1. What do you like best about teaching with the PLATO computer? 

Plato allows me to work one on one with students while it provides students with a 
learning tool that allows them to work at their own level and pace. 

2. What do you like least about teaching with the PLATO computer? 

When I encounter errors in the reporting system. 

3. Was there a regular time within the sequence of a lesson or unit in which your 
students experienced their PLATO modules? For example, did they visit the computer 
lab during the introduction to a new lesson or unit? Or did they use PLATO during the 
activity or information phase of a lesson? Or during the practice, review, or treinsfer 
(application) phase? Or was PLATO used only as an assessment tool? If yes, briefly 
explain. 

PLATO was offered for 45 minutes every other day to students working to meet state 
content standards. This was in addition to their regular math class. 

4. Describe any strategies you employed to determine whether or not the PLATO 
modules assigned to each student were the most appropriate for ensuring their success 
in your class. 

Aligned PLATO to state standards, assigned according to previous scores on state tests. 

5. How would you change the PLATO lessons? 

I would not 



6. What suggestions do you have to improve the way you use the PLATO system? 
Require applications and note taking to improve retention. 



7. What other comments or suggestions do you have on the PLATO system or this 
course? 

<No answer> 
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OSAT Gains by Strand 





Forest Grove High School 



Page 35 



34 - 



Algebraic Relations 


3.5- 
3 - 

2.5- 
2 - 

1.5- 
1 - 

0.5- 




3-.15 ” 


j 

1 

t 

1 


0 - 
-0.5 
-1^ 




■ 

Plato users » 


lon» Plato J 

-0.48 


Lers 

i 





Figure 3 



Geometry 




Figure 4 




Forest Grove High School 



Page 36 



35 



Measurement 




Figure 5 
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Figure 6 
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Figure 9 
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Percentile Rankings 



Plato users percentile rankings 




Figure 10 
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Non-Plato percentile rankings 
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FGHS students percentile rankings 




Figure 12 
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Appendix C; Learner Survey Results 



Questi 

on 




SA 


A 


N 


D 


SD 


n 


M 

(SD) 


1. 


I am able to sign on 
to the computer 
without problems. 


17 


17 


2 


1 


0 


37 


4.35 

(.716) 


2. 


Getting to my lesson is easy. 


14 


20 


3 


0 


0 


37 


4.30 

(.618) 


3. 


The computer is easy to use. 


13 


19 


5 


0 


0 


37 


4.22 

(.672) 


4. 


I can start and stop a lesson 
whenever I want. 


19 


11 


5 


1 


0 


36 


4.33 

(.828) 


5. 


The computer lets me do 
something (like answer 
questions) often and not 
mainly just watch. 


13 


18 


3 


2 


0 


36 


4.17 

(.811) 


6. 


"I usually can understand what 
the computer teaches me, 
without help from my 
instructor." 


6 


12 


10 


8 


1 


37 


3.38 

(1.09) 


7. 


The computer gives me help 
when I need it. 


5 


11 


15 


5 


1 


37 


3.38 

(.982) 


8. 


I can work at my own pace on 
the computer. 


11 


23 


2 


1 


0 


37 


4.19 

(.660) 


9. 


I'm studying the same topics at 
the same time on and off the 
computer. 


5 


12 


13 


7 


0 


37 


3.41 

(.956) 


10. 


My teacher helps me see the 
connection between what I'm 
studying on and off the 
computer. 


14 


13 


7 


2 


1 


37 


4.00 

(1.03) 
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11. 


I feel I'm studying what I need 
to on the computer. 


10 


15 


7 


5 


0 


37 


3.81 

(.996) 


12. 


The lessons on the computer 
are designed for people like 
me. 


6 


9 


14 


4 


2 


35 


3.37 

(1.09) 


13. 


"When I give a wrong answer 
on the computer, I feel bad 
about myself" 


1 


4 


14 


12 


6 


37 


2.51 

(.989) 


14. 


I would like more time to 
study on the computer. 


5 


5 


16 


7 


4 


37 


3.00 

(1.15) 


15. 


The computer makes me 
nervous. 


1 


1 


11 


11 


13 


37 


2.08 

(1.01) 


16. 


Working on the computer 
makes me feel good about 
myself 


3 


4 


21 


6 


3 


37 


2.95 

(.97) 


17 . 


I recommend learning from the 
computer. 


5 


11 


13 


4 


3 


36 


3.31 

(1.12) 


18. 


The computer lessons I work 
with are interesting. 


2 


13 


14 


5 


3 


37 


3.16 

(1.01) 


19. 


I try hard to learn from the 
computer lessons. 


5 


18 


13 


2 


0 


37 


3.73 

(.732) 


20. 


The computer lessons make 
me feel more confident about 
doing well in school. 


4 


13 


14 


5 


1 


37 


3.38 

(.953) 


21. 


Describe how much you think 
the PLATO computer system 
has helped you in Math. 














3.24 

(1.09) 



Forest Grove High School 



Page 44 




43 



1. What do you like best about learning from the computer? 



• Work at my own pace. [12] 

• No teachers involved. [4] 

• Give us extra help in the tutorial. [2] 

•No homework. [2] 

j 

•lam doing a lot more better in Math!!! [1] 

• What I like best about a computer is that shows you how to do the work. [1 ] 

• The best part is because all the work stays on the computer. [1] 

•You don't have to write anything. [1] 

• If you fail a test, you can go back to the tutorial and find out how to do the problems you missed. [1] 

• You can take the tests more than once if you don't pass the first time. [1] 

• I get a star when I complete the unit I'm in. [1] 

• It was more hands-on. [1] 

• I don't get embarrassed when I get the answer wrong. [1] 

• It is easy to cheat. [1] 

• I like how I can move from lesson to lesson whenever I want. [1] 

• That it explains first what to do before starting and review. [1] 



2. What do you like least about learning from the computer? 



• It is boring. [9] 

• Tutorial. [2] 

• That I can't figure anything out. [2] 

• Sometimes I don't understand. [2] 

• Can't have discussions with an inanimate object. [2] 

• The thing where if your variables are in capitals they mark it wrong. [1] 

• Sometimes it gets a little confusing. [1] 

• On mastery tests, you don't find out what it was that you missed. [1] 

• Taking mastery tests over and over. [1] 

• Can be doing something more valuable. [1] 

• Only when it doesn't work! [1] 
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• No homework, sometimes I just want homework. [1] 

• Sometimes, I don't understand the way they do the problems. [1] 

• That we can't go on mastery test whenever we want to. [1 ] 

• It not very easy to go back and do it again. [1 ] 

• I can't ask it questions, such as if I wanted to know more on how the xxx works. I can’t ask a 
computer that. [ 1 ] 

• Does not always let me know what I did wrong on a missed problem. [1] 

• That sometimes you don't even touch the computer and it logs as itself. [1] 

• It is not adjusted to the person; it is just one test for everyone. [1] 

• Sometimes it won’t work I don't understand. [1] 

• Makes me feel like a failure when I'm done. [1] 

• Too slow. [1] 

• It doesn't really help. [1] 

• Only one try and test answers and then it goes if xxx start over. [1] 



3. How would you change the computer lessons or the way you use them? 



• I would not change them at all. [8] 

• I would have them be a lot more interesting. [4] 

• No tutorial. 

• I don't think I can change. 

•Not have to do them that much. 

• Shorter. Not as much stuff to work on. 

• Speaks to you. 

• Yes, maybe. 

• Make them more for individual people. 

• Make it easier to learn the lessons. 

• I would make them more fun like and have a student, teacher level. 

• Higher math categories. 

• Give more helpful help. 
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4. What other suggestions do you have to improve any part of the course(s) which use PLATO? 



• Not have to be on the computers that long. [3] 

• Have more teachers in the classroom to help explain the problems and more time to do assignments 
(like more applications and practice more before the test). [1] 

• I think you should have more questions on the tests! [1] 

• I have no suggestions, everything is fine. [1] 

• Don't make it all gray in the things/mode. [1] 

• Even more stuff after finishing. [1] 

• Have something to read it for you. [1 ] 

• I don't think it needs improve when we got two very cool teachers in the lab to always help! [1] 



Graphs of Learner Survey Data 

Note: in these graphs, 1= Strongly Disagree, 2= Disagree, 3= Neither 
Agree nor Disagree, 4= Agree, 5= Strongly Agree 
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